Principles, Practice, and Pragmatics of Measurement in Experimental Computer Science

Grant number: DP110101934 | Funding period: 2011 - 2014

Completed

Abstract

The project team's confidence in scientific knowledge is partly due to the robustness of the systems of measurement used in experiments. The project aims to develop principles of measurement in Computer Science, where it is critically important that we have high confidence in the methods used in applications such as medicine, finance, and internet search.

Related publications (9)

Scholarly works icon

Users: Can't Work With Them, Can't Work Without Them?

Alistair Moffat

2022-07-06

If we could design the ideal IR "effectiveness" experiment (as distinct from an IR "efficiency" experiment), what would it look li..

Scholarly works icon

Can machine translation systems be evaluated by the crowd alone

Y Graham, T Baldwin, A Moffat, J Zobel

2017-01-01

Crowd-sourced assessments of machine translation quality allow evaluations to be carried out cheaply and on a large scale. It is e..

Scholarly works icon

Retrieval consistency in the presence of qery variations

P Bailey, A Moffat, F Scholer, P Thomas

2017-01-01

© 2017 Copyright held by the owner/author(s). A search engine that can return the ideal results for a person's information need, i..

Scholarly works icon

Pooled Evaluation Over Query Variations: Users are as Diverse as Systems

A Moffat, F Scholer, P Thomas, P Bailey

2015-10-01

Evaluation of information retrieval systems with test collections makes use of a suite of fixed resources: a document corpus; a se..

Scholarly works icon

Towards Nuanced System Evaluation Based on Implicit User Expectations

P Thomas, P Bailey, A Moffat, F Scholer

2015-01-01

Information retrieval systems are often evaluated through the use of effectiveness metrics. In the past, the metrics used have cor..

Scholarly works icon

User Variability and IR System Evaluation

P Bailey, A Moffat, F Scholer, P Thomas

2015-01-01

Test collection design eliminates sources of user variability to make statistical comparisons among information retrieval (IR) sys..

Scholarly works icon

Assessing the Cognitive Complexity of Information Needs

AM MOFFAT, P Bailey, F Scholer, P Thomas

2014-01-01

Information retrieval systems can be evaluated in laboratory settings through the use of user studies, and through the use of test..

Scholarly works icon

Automatic keyphrase extraction from scientific articles

SN Kim, O Medelyan, MY Kan, T Baldwin

2013-09-01

This paper describes the organization and results of the automatic keyphrase extraction task held at the Workshop on Semantic Eval..

Scholarly works icon

Word sense and semantic relations in noun compounds

SN Kim, T Baldwin

2013-07-01

In this article, we investigate word sense distributions in noun compounds (NCs). Our primary goal is to disambiguate the word sen..

University of Melbourne Researchers